[UIAM] Cloud API key authentication metadata and validations #129227

slobodanadamovic · 2025-06-10T21:50:58Z

A followup to #128440, which introduces a new managed_by field (<1>) that will be returned in the response of the Authenticate API.

Besides managed_by field, it also captures additional internal field (<2>) for cloud API key authentication and exposes it as part of the api_key fields.

{
  "username": "omSAd5YBK3gZiBcD-GvX", 
  "roles": [ "viewer" ],
  "metadata": {
    ...
  },
  "enabled": true,
  "authentication_realm": {
    "name": "_cloud_api_key",
    "type": "_cloud_api_key"
  },
  "lookup_realm": {
    "name": "_cloud_api_key",
    "type": "_cloud_api_key"
  },
  "authentication_type": "api_key",
  "api_key": { 
    "id": "omSAd5YBK3gZiBcD-GvX",
    "name": "my cloud API key",
    "managed_by": "cloud", <1>
    "internal": false <2>
  }
}

Additionally it implements the Authentication#canAccessResourcesOf for the cloud API keys. Ownership check allows access only to the same cloud API key.
And lastly, adds a consistency check for cloud API keys in Authentication#checkConsistencyForApiKeyAuthenticationType.

…ud-api-key-authentication

…ud-api-key-authentication # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java

…hentication-metadata-and-validation # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java

…hentication-metadata-and-validation # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java # x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Authentication.java # x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/AuthenticationField.java # x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authc/AuthenticationSerializationTests.java # x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/Security.java

…hentication-metadata-and-validation

elasticsearchmachine · 2025-06-17T22:02:07Z

Pinging @elastic/es-security (Team:Security)

n1v0lg

Same on this PR: looks good, but I want to take a bit of time in the morning to fully complete the review. Don't foresee big change requests, nice work 👍

A couple suggestions.

n1v0lg · 2025-06-18T15:09:20Z

...ck/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Authentication.java

-        }
-        checkConsistencyForApiKeyAuthenticatingSubject("API key");
+        final String prefixMessage = authenticatingRealm.isCloudApiKeyRealm() ? "Cloud API key" : "API key";
+        checkConsistencyForApiKeyAuthenticatingSubject(prefixMessage);


I'd prefer a separate method checkConsistencyForCloudApiKeyAuthenticatingSubject for cloud API keys here -- cloud API keys are fundamentally different from stack API keys. For example, we don't expect role descriptors in cloud API key metadata. We currently don't check for the presence of role descriptors for stack API keys either but could easily do that in the future. Similarly I think it's a good consistency check that cloud API keys don't have role descriptors in metadata.

I'd also add a check to checkConsistencyForApiKeyAuthenticatingSubject that internal is not set since that doesn't make sense for stack API keys.

I've pushed the changes which add dedicated consistency check for cloud API keys and additionally made run-as unsupported for cloud API keys. Currently, there is no need to support them. We can always enable it in the future.

n1v0lg · 2025-06-18T15:13:24Z

...ugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/AuthenticationField.java

    public static final String API_KEY_CREATOR_REALM_TYPE = "_security_api_key_creator_realm_type";
    public static final String API_KEY_ID_KEY = "_security_api_key_id";
    public static final String API_KEY_NAME_KEY = "_security_api_key_name";
+    public static final String API_KEY_MANAGED_BY_KEY = "_security_api_key_managed_by";


I don't think we need this metadata field -- we can always infer the value based on the subject type. Less is more in this case, IMO

Agreed. FWIW I initially started by inferring it, then realized we planned to have it for all authentication types. Since we are not requring it for all, just API keys (at the moment), I'll change this and remove it.

...core/src/test/java/org/elasticsearch/xpack/core/security/authc/AuthenticationTestHelper.java

...security/src/main/java/org/elasticsearch/xpack/security/audit/logfile/LoggingAuditTrail.java

...ck/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Authentication.java

...core/src/test/java/org/elasticsearch/xpack/core/security/authc/AuthenticationTestHelper.java

n1v0lg

Discussed on Zoom: slight leaning towards not storing api key ID in metadata since it enforces that the API key ID is the principal for cloud API keys and achieves consistency by definition instead of requiring explicit consistency checks (API key ID == principal for cloud API keys).

Looks good otherwise!

...ck/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Authentication.java

...ugin/core/src/test/java/org/elasticsearch/xpack/core/security/authc/AuthenticationTests.java

…hentication-metadata-and-validation

n1v0lg

LGTM 🚀

…#129227) A followup to elastic#128440, which introduces a new `managed_by` field (`<1>`) that will be returned in the response of the Authenticate API. Besides `managed_by` field, it also captures additional `internal` field (`<2>`) for cloud API key authentication and exposes it as part of the `api_key` fields. ```json { "username": "omSAd5YBK3gZiBcD-GvX", "roles": [ "viewer" ], "metadata": { ... }, "enabled": true, "authentication_realm": { "name": "_cloud_api_key", "type": "_cloud_api_key" }, "lookup_realm": { "name": "_cloud_api_key", "type": "_cloud_api_key" }, "authentication_type": "api_key", "api_key": { "id": "omSAd5YBK3gZiBcD-GvX", "name": "my cloud API key", "managed_by": "cloud", <1> "internal": false <2> } } ``` - Additionally it implements the `Authentication#canAccessResourcesOf` for the cloud API keys. Ownership check allows access only to the same cloud API key. - And lastly, adds a consistency check for cloud API keys in `Authentication#checkConsistencyForApiKeyAuthenticationType`.

n1v0lg and others added 30 commits May 26, 2025 10:21

[UIAM] Cloud API key authentication

d5bc9d2

Clean up

c673649

Nit

3f6b6ff

Merge branch 'main' into uiam-cloud-api-key-authentication

604c630

Fix more tests

95c9a38

Nit

d45fe0c

Merge branch 'main' into uiam-cloud-api-key-authentication

cd8b9f1

Fix sig

3be47f0

Merge branch 'main' into uiam-cloud-api-key-authentication

12908fa

Fix not

0b6bdff

Nit

c974761

Merge branch 'main' into uiam-cloud-api-key-authentication

113f6a5

Merge branch 'main' into uiam-cloud-api-key-authentication

5b89907

Merge branch 'main' into uiam-cloud-api-key-authentication

7bfb559

Authenticator

6966cea

More

e3abd81

Javadoc

8b0f1d3

Javadoc

ca6efe8

Fix tests

444b9a1

Exception handling

f868daf

Javadoc

e4f5b9e

Merge branch 'main' into uiam-cloud-api-key-authentication

0686c92

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-clo…

65aebd2

…ud-api-key-authentication

add new transport version

f1965d3

add todo to followup in ES-11961

30dc57d

test cloud API key authentication serialization

bd19d18

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-clo…

4d07cdc

…ud-api-key-authentication # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java

Add cloud API key metadata and managed_by to authentication

e32af54

reuse metadata keys and add consistency check

d17cc23

support audit logging (pending tests)

999ac5c

elasticsearchmachine added v9.1.0 serverless-linked Added by automation, don't add manually labels Jun 10, 2025

slobodanadamovic and others added 8 commits June 11, 2025 09:10

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-aut…

4cf1ad7

…hentication-metadata-and-validation # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java

fix serialization tests

ddb73d4

define managed_by under api_key field

e796769

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-aut…

9aa18de

…hentication-metadata-and-validation

[CI] Auto commit changes from spotless

9495904

adjust authentication tests

bc55bfb

fix failing tests

8ff2b6d

slobodanadamovic requested review from n1v0lg and tvernum June 17, 2025 22:01

slobodanadamovic marked this pull request as ready for review June 17, 2025 22:01

n1v0lg reviewed Jun 18, 2025

View reviewed changes

n1v0lg reviewed Jun 19, 2025

View reviewed changes

...ck/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Authentication.java Outdated Show resolved Hide resolved

...ugin/core/src/test/java/org/elasticsearch/xpack/core/security/authc/AuthenticationTests.java Outdated Show resolved Hide resolved

slobodanadamovic and others added 7 commits June 20, 2025 17:19

address review feedback

740b073

[CI] Auto commit changes from spotless

1455346

improve authorization denial messages for cloud API keys

3e4099c

make run-as unsupported for cloud api keys

532e8ff

move run-as consistency check

52348f6

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-aut…

667710d

…hentication-metadata-and-validation

cleanup after refactoring

ea11ed1

slobodanadamovic requested a review from n1v0lg June 23, 2025 07:33

Merge branch 'main' of github.com:elastic/elasticsearch into uiam-aut…

217b48c

…hentication-metadata-and-validation

n1v0lg approved these changes Jun 23, 2025

View reviewed changes

slobodanadamovic merged commit 2502a36 into elastic:main Jun 23, 2025
39 checks passed

[UIAM] Cloud API key authentication metadata and validations #129227

[UIAM] Cloud API key authentication metadata and validations #129227

Uh oh!

Conversation

slobodanadamovic commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Jun 17, 2025

Uh oh!

n1v0lg left a comment

Choose a reason for hiding this comment

Uh oh!

n1v0lg Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

slobodanadamovic Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

slobodanadamovic Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

n1v0lg Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

slobodanadamovic Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

n1v0lg left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

n1v0lg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

slobodanadamovic commented Jun 10, 2025 •

edited

Loading

n1v0lg left a comment •

edited

Loading